Insensitivity of place cells to the value of spatial goals in a two-choice flexible navigation task

Hippocampal place cells show position-specific activity, thought to reflect a self-localization signal. Several reports also point to some form of goal encoding by place cells. We investigated this by asking whether they also encode the value of spatial goals, which is a crucial information for optimizing goal-directed navigation. We used a continuous place navigation task in which male rats navigate to one of two (freely chosen) unmarked locations and wait, triggering the release of reward which is then located and consumed elsewhere. This allows sampling of place fields, and dissociates spatial goal from reward consumption. The two goals varied in the amount of reward provided, allowing assessment of whether the rats factored goal value into their navigational choice, and of possible neural correlates of this value. Rats successfully learned the task, indicating goal localization, and they preferred higher-value goals, indicating processing of goal value. Replicating previous findings, there was goal-related activity in the out-of-field firing of CA1 place cells, with a ramping-up of firing rate during the waiting period, but no general over-representation of goals by place fields, an observation that we extended to CA3 place cells. Importantly, place cells were not modulated by goal value. This suggests that dorsal hippocampal place cells encode space independently of its associated value, despite the effect of that value on spatial behavior. Our findings are consistent with a model of place cells in which they provide a spontaneously constructed value-free spatial representation, rather than encoding other navigationally relevant, but non-spatial, information

Electrophysiological correlates of high-level perception during spatial navigation

We studied the electrophysiological basis of object recognition by recording scalp\ud electroencephalograms while participants played a virtual-reality taxi driver game.\ud Participants searched for passengers and stores during virtual navigation in simulated\ud towns. We compared oscillatory brain activity in response to store views that were targets or\ud nontargets (during store search) or neutral (during passenger search). Even though store\ud category was solely defined by task context (rather than by sensory cues), frontal ...\ud \u

Pointing errors in non-metric virtual environments

There have been suggestions that human navigation may depend on representations that have no metric, Euclidean interpretation but that hypothesis remains contentious. An alternative is that observers build a consistent 3D representation of space. Using immersive virtual reality, we measured the ability of observers to point to targets in mazes that had zero, one or three ‘wormholes’ – regions where the maze changed in configuration (invisibly). In one model, we allowed the configuration of the maze to vary to best explain the pointing data; in a second model we also allowed the local reference frame to be rotated through 90, 180 or 270 degrees. The latter model outperformed the former in the wormhole conditions, inconsistent with a Euclidean cognitive map

Spatial Learning and Action Planning in a Prefrontal Cortical Network Model

The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates

Whisker Movements Reveal Spatial Attention: A Unified Computational Model of Active Sensing Control in the Rat

Spatial attention is most often investigated in the visual modality through measurement of eye movements, with primates, including humans, a widely-studied model. Its study in laboratory rodents, such as mice and rats, requires different techniques, owing to the lack of a visual fovea and the particular ethological relevance of orienting movements of the snout and the whiskers in these animals. In recent years, several reliable relationships have been observed between environmental and behavioural variables and movements of the whiskers, but the function of these responses, as well as how they integrate, remains unclear. Here, we propose a unifying abstract model of whisker movement control that has as its key variable the region of space that is the animal's current focus of attention, and demonstrate, using computer-simulated behavioral experiments, that the model is consistent with a broad range of experimental observations. A core hypothesis is that the rat explicitly decodes the location in space of whisker contacts and that this representation is used to regulate whisker drive signals. This proposition stands in contrast to earlier proposals that the modulation of whisker movement during exploration is mediated primarily by reflex loops. We go on to argue that the superior colliculus is a candidate neural substrate for the siting of a head-centred map guiding whisker movement, in analogy to current models of visual attention. The proposed model has the potential to offer a more complete understanding of whisker control as well as to highlight the potential of the rodent and its whiskers as a tool for the study of mammalian attention

Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail

Changes of synaptic connections between neurons are thought to be the physiological basis of learning. These changes can be gated by neuromodulators that encode the presence of reward. We study a family of reward-modulated synaptic learning rules for spiking neurons on a learning task in continuous space inspired by the Morris Water maze. The synaptic update rule modifies the release probability of synaptic transmission and depends on the timing of presynaptic spike arrival, postsynaptic action potentials, as well as the membrane potential of the postsynaptic neuron. The family of learning rules includes an optimal rule derived from policy gradient methods as well as reward modulated Hebbian learning. The synaptic update rule is implemented in a population of spiking neurons using a network architecture that combines feedforward input with lateral connections. Actions are represented by a population of hypothetical action cells with strong mexican-hat connectivity and are read out at theta frequency. We show that in this architecture, a standard policy gradient rule fails to solve the Morris watermaze task, whereas a variant with a Hebbian bias can learn the task within 20 trials, consistent with experiments. This result does not depend on implementation details such as the size of the neuronal populations. Our theoretical approach shows how learning new behaviors can be linked to reward-modulated plasticity at the level of single synapses and makes predictions about the voltage and spike-timing dependence of synaptic plasticity and the influence of neuromodulators such as dopamine. It is an important step towards connecting formal theories of reinforcement learning with neuronal and synaptic properties

Spatial memory in the grey mouse lemur (Microcebus murinus)

Wild animals face the challenge of locating feeding sites distributed across broad spatial and temporal scales. Spatial memory allows animals to find a goal, such as a productive feeding patch, even when there are no goal-specific sensory cues available. Because there is little experimental information on learning and memory capabilities in free-ranging primates, the aim of this study was to test whether grey mouse lemurs (Microcebus murinus), as short-term dietary specialists, rely on spatial memory in relocating productive feeding sites. In addition, we asked what kind of spatial representation might underlie their orientation in their natural environment. Using an experimental approach, we set eight radio-collared grey mouse lemurs a memory task by confronting them with two different spatial patterns of baited and non-baited artificial feeding stations under exclusion of sensory cues. Positional data were recorded by focal animal observations within a grid system of small foot trails. A change in the baiting pattern revealed that grey mouse lemurs primarily used spatial cues to relocate baited feeding stations and that they were able to rapidly learn a new spatial arrangement. Spatially concentrated, non-random movements revealed preliminary evidence for a route-based restriction in mouse lemur space; during a subsequent release experiment, however, we found high travel efficiency in directed movements. We therefore propose that mouse lemur spatial memory is based on some kind of mental representation that is more detailed than a route-based network map

Hippocampal and prefrontal processing of network topology to simulate the future

Topological networks lie at the heart of our cities and social milieu. However, it remains unclear how and when the brain processes topological structures to guide future behaviour during everyday life. Using fMRI in humans and a simulation of London (UK), here we show that, specifically when new streets are entered during navigation of the city, right posterior hippocampal activity indexes the change in the number of local topological connections available for future travel and right anterior hippocampal activity reflects global properties of the street entered. When forced detours require re-planning of the route to the goal, bilateral inferior lateral prefrontal activity scales with the planning demands of a breadth-first search of future paths. These results help shape models of how hippocampal and prefrontal regions support navigation, planning and future simulation